Fermions in optical lattices near a Feshbach resonance: 
from band insulator to Mott insulator 
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We study a model of an equal mixture of two species of fermions in a deep optical lattice at a 
filling of two fermions per site. At weak inter-species interaction, the system is a band insulator. 
When the inter-species interaction is tuned via a Feshbach resonance to be larger than an energy 
related to the energy separation of the first and second Bloch band, atoms populate equally the two 
Bloch bands. With weak tunneling between sites of the optical lattice, the system becomes a Mott 
insulator with the low energy effective Hamiltonian of a spin-1 Heisenberg antiferromagnet, because 
of a Hund's rule like coupling between the two bands. We discuss experimental signatures of these 
two types of insulators. 

PACS numbers: 03.75.Ss, 03.75.Mn, 71.10.Fd, 05.30.Fk 



The controllability and the capability to continuously 
tune parameters has provided unprecedented opportuni- 
ties to study strong correlation physics in trapped ultra 
cold atoms systems. Thus, Greiner et a/.Q] observed the 
quantum phase transition from a superfluid to a Mott 
insulator for bosons trapped in an optical lattice, by sup- 
pressing tunneling between sites of the lattice. Recently, 
there has been much experimental progress in study- 
ing fermionic superfluidity in the BCS to BEC crossover 
regime Q whereby the scattering length between two 
species of fermions is continuously tuned through a Fes- 
hbach resonance. 

In an optical lattice, the hopping matrix element t and 
the on-site interaction strength U both depend essen- 
tially on the amplitude Vo of the laser beams that de- 
fine the optical lattice. But by adding an appropriate 
magnetic field to sweep through a Feshbach resonance, 
a larger range of U vs. t can be accessed experimen- 
tally. Very recently, Kohl ef.aZ.|j| have exploited this to 
study two hyperfine states of fermionic 40 K (about 50:50 
mixture) loaded into a three dimensional (3D) optical 
lattice. Initially, far from the Feshbach resonance (i.e. 
no interaction), the lattice is loaded with two fermions 
per site (one for each spin state) into the lowest Bloch 
band, with weak tunneling between sites. Hence the sys- 
tem is a band insulator. Now turning up the magnetic 
field towards the inter-species Feshbach resonance causes 
U to increase to bigger than the interband energy, hence 
they measure atoms being kicked into upper Bloch bands. 
The key question^ is this: what happens to the band 
insulator as the Feshbach resonance is approached? 

For atoms with Feshbach resonance enhanced inter- 
actions, Diener and Ho0 have shown in the single site 
problem, and Katzgraber ei.aL0 in the full lattice prob- 
lem, that there can be a window of magnetic field where 
essentially two Bloch bands are occupied: the harmonic 
oscillator ground state [000], together with the [001] band 
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FIG. 1: Schematic phase diagram. The middle row shows 
the dominant spatial structure of the two species ("spin") of 
fermions in orbital and 1; the top row shows the schematic 
correlation function G(r, r') that may be measured in noise 
correlation experiments: the Mott insulator has extra peaks 
due to doubling of real space unit cell. 



for an anisotropic lattice (as in the experiment of 0). At 
or very near the Feshbach resonance itself, more bands 
may be occupied. 

Thus, motivated by the experiment of [f|, we study 
in this paper a simple model of spin- 1/2 fermions in an 
optical lattice at a filling of two fermions per site, with 
equal mix of the two spin species, and where two Bloch 
bands are active, as a function of U : 



a, (3. a,i 



-tr 



4+i* a c i<?P +H.c. 

Oil 



OLIO 

£ 



(1) 



a "'KTCk. 



ALA 



i/3 



where the band index a, j3 = 0, 1 for the [000] and [001] 
bands, and the "spin" 8] index a =|, j, and the site index 
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» S ta = c ha c Ha, and A l0 = c iip c iW . 
The chemical potential difference fi — \i\ — lu is the dif- 
ference in energy between the two bands. For simplicity, 
we here consider only a ID system. This is achieved in 
the experiments of ,9j by setting the laser amplitudes 
Vq x <C Vq v — Vq z , thereby creating an uncoupled set 
of ID tubesQ, where excitations transverse to the tube 
axis are completely frozen. For higher dimensions, be- 
cause of the spatial anisotropy of the [001] orbital, the 
hopping matrix element t a p acquires spatial anisotropy 
also which complicates analysis. | lOllll| 

This Hamiltonian has a similar form to those in solid 
state systems when multiple orbitals are active in eg. 
transition metal oxides 0: the — S + S~ term gives rise 
to the Hund's rule in atomic physics, that favours spin 
alignment between different orbitals. The difference here 
in the optical lattice is that the atomic interactions are 
contact interactions, not the long range Coulomb interac- 
tions for electrons in solids. Hence, the Hund's rule term 
is of the same order of magnitude as the on-site repulsion 
term, unlike in solids. Also unlike in solids, experimental 
preparation^ dictates that the number of fermions for 
each spin species is conserved separately. 

In this paper, we study this model Eq. ^ at weak in- 
tersite tunneling and at zero temperature. We show that 
when the Feshbach-tunable interactions are weak com- 
pared to the inter-band energy u)q, a "spin" -singlet [8| of 
two fermions reside in each lattice well in the lowest Bloch 
band to form a band insulator. When the interactions 
are strong relative to ujq, the two fermions at each site 
reside in different bands with aligned "spin" forming an 
effective spin-1. This effective spin-1 alternates in sign 
between neighbouring sites to take advantage of second 
order virtual hopping of the fermions, thereby forming 
a spin-1 Heisenberg antiferromagnet, see FigQ This is 
a correlated (Mott) insulator state as each band is only 
half-filled, and is reminiscent of the charge-transfer in- 
sulator in transition- metal compounds |l2j. and excitonic 
insulators 0. 

The use of Feshbach resonance to populate higher 
Bloch bands of optical lattices promises to open up 
much interesting orbital physics that may go beyond 
those models studied in the context of transition-metal 
oxides^ij. For example, in 2D and 3D, there can be or- 
bital spatial ordering in a lattice with a single species of 
bosons^. Duan has also studied the effect of Fes- 
hbach resonance on fermions in optical lattices, but as 
that study was in a different regime, he found effective 
models with spin- 1/2 antiferromagnetic correlations. 
Model derivation The microscopic ID Hamiltonian is: 
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Since we are interested in the regime where at most 
two Bloch bands are occupied (roughly, U < 2ojq), 
we expand the operator ^ a {x) in Wannier functions: 
ipa{x) — J2 a =o \ w ia{x)ciaa , where for a deep optical 
lattice, we approximate the Wannier functions by lo- 
cal harmonic oscillator orbitals. I shall consider band 
to be the ground state: Wio(x) 



3wi exp 



and with band 1 to be the first excited state: wn (x) = 
i ^~2u/4 ^ x ~ Xi) exp ^ x ~ x $ . Note that these functions, 
unlike Wannier functions, are not orthogoal for different 
bands with respect to the hopping, hence there can be 
band-changing hopping with t m = t w . In fact, in general 
for band 1 to have a higher energy, |*oo | < |*oi I < |*n|- 

Then substituting the approximate Wannier functions 
into Eq. [5] gives the Hamiltonian of Eq. ^ with the pa- 
rameters: Uqo = cqoU, Uqi — cqiU, U\x — ciiU, U — 
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W (€) ^' V = ( V °* V 0yVaz) 1/3 ■ (We are giving 
the values for the actual 3D situation wherein the ID 
tubes are embedded, i.e. Vq x <§; Vq v = Vo 2 .;9j) a s is the 
Feshbach resonance enhanced effective scattering length 
between the species, which can be tuned by magnetic 
field. In general a s will depend on the energy of the 
two scattering particles, and is renormalized from the 
free space value by strong transverse confinement and the 
Feshbach resonance. In absence of calculations, we have 
left as parameters c a p to denote this band dependence. 
As a crude guide, treating a s as energy- independent, 
coo = l,coi = 1/2, en = 3/4, and a s refers then to 
some average value, but with Feshbach and transverse 
confinement renormalization. We only consider the situ- 
ation where the broad Feshbach resonance is approached 
from the side where the scattering length is positive, 
with few if any molecules formed. Hence sweeping the 
magnetic field towards the Feshbach resonance effectively 
makes the couplings U a /3 cx g — 47ra s /M > grows from 
U a p <C u>o t° U a /3 <; luq. We have approximated the inter- 
band energy ujq to be just the harmonic oscillator energy 
in the x direction: uj ps y/AVo x E r with E r = k 2 /2M the 
recoil energy for the optical lattice, k — 2tt/X where A is 
the laser wavelength, with the lattice spacing a = A/2. 
Strong coupling analysis Kohl et.aVs experiment is in 
the strong coupling limit, so we focus on t a p <C u>o, U a $- 
Thus, first diagonalise the Hamiltonian Eq.^with t a p — 
to get the local spectrum with two fermions (one of 
each spin) per site: 
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the spin-1 Heisenberg antiferromagnet: 
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FIG. 2: Probability of a singlet in band (Pq: upper curve) 
or band 1 (Pi: lower curve) as a function of U/ljo, for the 
eigenstate |s+). The vertical line shows the location of U c . 
Note that |s+) is the ground state only for U < U c , and for 
for U > 2ujq, higher bands start to be occupied. 
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oi|U;0) + a 3 |0;U) 
O2|U;0)-ai|0;U) 



The notation for the eigenstates is that the left most slot 
is for band 0, separated by a ; from the band 1 slot. (We 
have dropped a site index on the eigenstates.) 01,02 are 
functions of U a p and loq with a\ + a 



plot the probability of a singlet in band or band 1, using 
c 00 = 1, c i = 1/2, en = 3/4. 

(s) 

Since U a p > 0, the lowest energy is E\' for U a p < 
U°g, or when U a p > U° g . The two levels cross 
at U£,p(wo) given by the implicit equation: (U^) 2 = 
(Uqo — ^o) (U n + ujo) ■ Note that there is only one tran- 
sition, since all the U a p oc U. Right at the transi 
tion, the parameters ai,a2 have the simple form: a\ 

~ {vtwfS'^ a2 = (fet) 1/2 ■ Thus ' when u <* < 

U^ g , at each site, the ground state is \s+) with a singlet 
in band mixed in with a bit of the singlet in band 1, 
while for U a p > U°g, the ground state is a spin triplet 
with one fermion in each band: |0) = \t+). Note that this 
triplet is degenerate with the other two members of the 
spin triplets: \U) = | T ; T), \D) = | 1 ; J.). In Kohl et aVs 
experiment, when the inter-species interaction is gradu- 
ally turned up, the states |J7), \D) cannot be reached at 
any sites if there are no hopping at all between the sites. 

When the hopping is switched on, the effective low- 
energy Hamiltonian can be derived using the usual 
strong coupling expansion to 0{t 2 /U), where interme- 
diate states have a site and its neighbour having three 
and one fermions respectively. We find^lj that for 
U a p < U^,p, the system is still a band insulator, while 
for U a p > U£,p, the system is now a Mott (correlated) 
insulator, with interesting spin dynamics which is that of 



U 00 + U 01 ~ujo U 11 + U 01 +oj 



where S is a spin-1 operator. Spin-1 is involved because 
of the spin degeneracy of the triplet states. Unlike the ID 
spin-1/2 antiferromagnet which has no spin gap, the spin- 
1 case in ID has a gap ~ 0.41J (the Haldane gap!9, 20]). 

At U a g = U°g, four states (the spin-1 triplet and \t+)) 
are now degenerate locally. The effective Hamiltonian at 
the transition point correpsonds to a spin ladder, where 
each leg is a spin-1/2 Heisenberg antiferromagnet, with 
exchange couplings between the sites on a rung of the 
ladder (i.e. S^S^, a is now the ladder leg index), but 
also diagonally (i.e. S^S^ V where 1,2 labels the two 
legs) . Full details will be published elsewhere. 01 We just 
point out here that generically, such spin ladders have a 
(3) spin gap^3^3- Hence it is likely that over the whole 
phase diagram as a function of U, the spin gap persists. 

So far, we have focussed on the large U limit and found 
only insulators. Clearly, when t a g ^> o>o, U a p, the system 
will be a metal. Thus in the whole phase diagram there 
1. In Fig|2 we should be metal-insulator transitions together with some 
multicritical point. Work is in progress to explore this 
rich phase diagram. 

Experimental signatures Both the band insulator and the 
Mott insulator have an energy gap to "charge" excita- 
tions, and interestingly, also for "spin" excitations. How- 
ever, for this band to Mott insulator quantum phase tran- 
sition, there are spin gaps (and also charge gaps) all the 
_ way from U a p < U^p to U a p > U°g (except perhaps 
right at the transition?), so a spin gap (as might be mea- 
sured via 2-photon Raman transitions |2ll l22|) does not 
qualitatively distinguish the band vs. the Mott insula- 
tor. The smoking gun experiment would be to image the 
"up" spins populating one sublattice while the "down" 
spins populate the other. 

But even without direct spin and site dependent 
imaging, current experimental probes such as noise 
correlation!^, 13 from time-of-fiight (TOF) imaging 
may distinguish between the band insulator and the 
Mott insulator. The ID momentum distribution can 
be readily evaluated, assuming that deep in the insu- 
lator phases, "charge" fluctuations are frozen. Thus, 
for the band insulator phase where most weight is in 
the band 0, n(Q r ) ~ Po\wo(Q r )\ 2 , while for the Mott 
insulator, with one fermion in each band at each site, 
n(Qr) ~ Po{\wa{Q r )\ 2 + \w 1 {Q r )\ 2 ) /2. Q r = Mr jt is 
the continuum momentum with r the imaging position 
relative to the initial distribution of cold atoms, and t is 
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the time of flight. This change in the filling of the bands 
may have been observed in [5j . 

To probe the "spin" spatial structure, one has to 
go to the noise correlation in the TOF imaging, 
which measures G(r,r') = 'Y^ (T(T i(n <T (Q r )n (T i{Q r i)) — 
{n<j(Q r )) {n-a' (Qr 1 )) [23( A straightforward generalization 
to the two band case gives a number of terms that 
show differences between the two types of insulators, but 
the key signature comes from the static spin structure 
factor |23j part within G(r,r'). Because of the different 
spatial symmetry of the Wannier functions of the two 
bands, G(r, r') does not directly probe the effective spin- 
1 correlations of the Mott insulator, but instead has in- 
formation on individual band spin correlations: 



G(r,r') 
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X wl(Q r )wp(Qr)Wp(Qr>)w a (Q r ') 

where S an is a spin-1/2 operator of band a at site 
r n = na. (The • • • refer to other terms without 
lattice periodicity mentioned above, we shall hence- 
forth ignore them.) For the band insulator, since 
at each site there is a singlet in band 0, there is 
no spin-spin correlation contribution, and G(r, r') oc 

-±K(Qr)| 2 MQrOI 2 Eg S {Q r - Qv + G), where G is 
a reciprocal lattice vector (integer multiples of2n/a). For 
the Mott insulator, assuming equal spin-spin correlation 
for the two bands and between bands, 
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G 

X *^2 W a(Qr)Wf3{Q r )Wp(Q r >)w a (Q r/ ). (7) 

a/3 

Thus, just as for the spin-1/2 Heisenberg antiferromag- 
net, a sharp (only relatively so in ID) peak of form f(Q) 
(see eg. |23) occurs in between the Bragg peaks at G, 
thanks to the doubling of the unit cell in real space for 
the antiferromagnet, see Fig^ The appearance of this 
extra peak constitutes proof of the spin structure, while 
the existence of a spin gap distinguishes between a spin-1 
from a spin-1/2 antiferromagnet. 

We have assumed so far that the system is homoge- 
neous and large. In experiments^, the system consists 
of around ~ 10 5 fermions in a few thousand tubes, and 
the overall trapping potential leads to inhomogeneities in 
occupation per site across the tube. This inhomogeneity 
can lead to phase coexistence j2^| of a Mott insulator sur- 
rounded by a shell of superfluidity for the bosonic Hub- 
bard model. Nevertheless, the bosonic Mott insulator 



has been observed 0, 0] , an d we expect the same to be 
possible for the fermionic model here, as long as there is a 
large enough central region with commensurate filling of 
two fermions per site. Also, in the experiment of the 
number of "spin-up" fermions equal the "down" ones up 
to a few percent, and the spin-1 antifcrromagnetic struc- 
ture described above should persist in the Mott phase. 
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